Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port improvements from ParallelProcessingTools #19

Merged
merged 7 commits into from
Apr 8, 2025
Merged

Conversation

oschulz
Copy link
Collaborator

@oschulz oschulz commented Mar 28, 2025

No description provided.

@oschulz oschulz marked this pull request as draft March 28, 2025 09:05
@oschulz oschulz mentioned this pull request Mar 28, 2025
15 tasks
oschulz added 3 commits March 28, 2025 14:54
Some use cases need to call elastic_worker, so it should be part of the
API.

Also rename stdout_to_master to forward_stdout, it's clearer.
Non-breaking since stdout_to_master was never documented.
@oschulz oschulz marked this pull request as ready for review March 28, 2025 13:57
@oschulz oschulz requested a review from DilumAluthge March 28, 2025 13:58
@DilumAluthge
Copy link
Member

Tbh I'm not terribly familiar with the ElasticManager code. @Moelf Would you be able to review?

The name of the field is confusing. It's much cleaner for the users to call
`get_connect_cmd` directly. Workers may be added in different ways to the same
ElasticManager and require a different startup. Also, `get_connect_cmd` was not
using printing_kwargs for it's kwargs by default, which was inconsistent.
Copy link

codecov bot commented Mar 30, 2025

Codecov Report

Attention: Patch coverage is 88.23529% with 2 lines in your changes missing coverage. Please review.

Project coverage is 91.75%. Comparing base (b3e8cb2) to head (c204136).
Report is 7 commits behind head on master.

Files with missing lines Patch % Lines
src/elastic.jl 88.23% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #19      +/-   ##
==========================================
- Coverage   93.33%   91.75%   -1.59%     
==========================================
  Files           2        2              
  Lines          90       97       +7     
==========================================
+ Hits           84       89       +5     
- Misses          6        8       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@oschulz
Copy link
Collaborator Author

oschulz commented Mar 30, 2025

Test are fixed now.

@oschulz
Copy link
Collaborator Author

oschulz commented Mar 31, 2025

Still tracking down an issue with persistent tasks.

@oschulz
Copy link
Collaborator Author

oschulz commented Apr 5, 2025

Still tracking down an issue with persistent tasks.

Still trying to figure out what's going on there ...

oschulz added 2 commits April 8, 2025 16:15
These are owned by Distributed, there's no reason to export them
from ElasticClusterManager.
Removing ElasticManager.printing_kwargs may be considered breaking,
it only affects show(::ElasticManager) but it was documented in
the README.
@oschulz oschulz force-pushed the ppt-improvements branch from 1851da5 to c204136 Compare April 8, 2025 14:15
@oschulz
Copy link
Collaborator Author

oschulz commented Apr 8, 2025

Still tracking down an issue with persistent tasks.

It looks like that was just a local artifact of Aqua testing on ParallelProcessingTools.

@oschulz
Copy link
Collaborator Author

oschulz commented Apr 8, 2025

For future reference, Aqua.test_persistent_tasks(SomePackage) reports a "persistant tasks" error when the local dev version of SomePackage depends on an unreleased version of SomeOtherPackage , even if that package is also in "dev-mode" and it has the new version locally - apparently Aqua.test_persistent_tasks runs an independent dependency resolution against the registry as part of what it does, and the fact that it then fails to resolve versions results in a "persistent task". Which normally wouldn't have been so confusing, but when testing a package whose tests involve running parallel tasks ...

@oschulz oschulz merged commit c204136 into master Apr 8, 2025
5 of 7 checks passed
@oschulz oschulz deleted the ppt-improvements branch April 8, 2025 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants